2D1431 Machine Learning Lab 4: Reinforcement Learning

In this lab you will learn about dynamic programming and reinforcement learning. It is assumed that you are familiar with the basic concepts of reinforcement learning and that you have read chapter 13 in the course book Machine Learning (Mitchell, 1997). The first four chapters of the survey on reinforcement learning by Kaelbling et al. (1996) is a good supplementary material. For further reading and a detailed discussion of policy iteration and reinforcement learning, the textbook “Reinforcement Learning” is highly recommendable (Sutton and Barto, 1999). In particular studying chapters 3,4 and 6 is of immense help for this lab. The predefined Matlab functions for this lab are located in the course directory /info/mi03/labs/lab4. Dynamic programming refers to a class of algorithms that can be used to compute optimal policies given a complete model of the environment. Dynamic programming solves problems that can be formulated as Markov decision processes. Unlike in the reinforcement learning case, dynamic programming assumes that the state transition and reward functions are known. The central idea of dynamic programming and reinforcement learning is to learn value functions, which in turn can be used to identify the optimal policy.